1. Field of the Invention
The present invention relates generally to image/video display and processing systems, and in particular, to a method, apparatus, and article of manufacture for simulating/reproducing film grain from a background image/clip in a new image/clip.
2. Description of the Related Art
Movies of still pictures shot using convention film often look grainy due to various physical and chemical aspects of the image formation and development processes. In film and photo compositing, it is sometimes desirable to simulate film grain. For example, when inserting computer animated content to a “real” image, grain can be artificially added to the computer generated image to create a more realistic effect. To create a seamless composition, the simulated film grain must match the grain present in the background “real”/reference image. Prior art methods require significant user input with respect to identifying the grain in the reference image. In addition, significant user interaction is required to reproduce/match/simulate the identified grain in the reference image. What is needed is the ability to easily and automatically (e.g., without significant user input) identify the grain in a reference image and to reproduce/match/simulate such an identified grain in a different image. These problems may be better understood with an explanation of prior art film grain simulation.
Different techniques may be utilized for adding grain to a background image. The H.264/MPEG-4 AVC describes an international video coding standard for use in a variety of applications (e.g., video telephony, television, storage, streaming video, etc.) that has been published by the International Telecommunication Union, Telecommunication Standardization Sector (a.k.a. ITU-T). Extensions to the H.264 standard provide various enhancements. A grain simulation mechanism is one such enhancement that was provided in the Fidelity Range Extensions (FRExt) Amendment 1. The grain simulation mechanism describes a film grain supplemental enhancement information (SEI) message that defines film grain characteristics such as size and intensity. The SEI allows a model of film grain statistics to be sent along with video data, that enables an analysis-synthesis style of video enhancement wherein a synthesized film grain is generated as a post-process when decoding, rather than requiring the encoder to determine and represent the exact film grain during the encoding process.
Thus, the SEI allows one to merely use a grain having the attributes specified in an SEI. While such a capability may be useful, it lacks the flexibility and ability to determine the grain (and the attributes/properties of such a grain) in a reference image where such an SEI does not exist for the reference image (e.g., the clip/film is scanned in or provided without the SEI or not in compliance with the H.264 specification). Further, the H.264 specification provides parameters and options that are not necessary to render realistic film grain effects. For example, photographic film grain aspects are dependent on exposure. Accordingly, it is important that grain parameters vary according to pixel color. The H.264 specification provides such a capability with the possibility to define up to five intensity intervals, each having a distinct set of parameters. The bounds of the intervals are user-defined, and each color channel has its own interval definitions. The use of intervals may be used to simplify the computational requirements and memory consumption of the grain generation algorithm. However, such a data representation is not user friendly.
In addition to the above problems, to identify a grainy area of a reference image, the prior art requires manual intervention. In this regard, the user is required to identify a series of regions in the reference image that correspond to smooth featureless areas. For example, the user must manually specify or identify a rectangular region of the reference image that contains the desired grain to be simulated. Such manual intervention is time consuming and requires the user to learn how to manipulate and identify the grain depending on the program being utilized.